File compression is a process that encodes information in an original file so that the information in the original file could be expressed with fewer bits. Compression is very useful because it helps to reduce the consumption of expensive resources, for example, hard disk space while storing information, transmission bandwidth while transmitting information, etc. Due to the demand of reducing the consumption of resources, the compression technique has been developed quickly, and a lot of compression algorithms have been proposed. Different compression formats are utilized in different compression algorithms, such as ZIP, RAR format in Microsoft® Windows® operating system, gz and bz format for Linux®/Unix® operating system.
Multiple files can be compressed into a single file by file compression. However, when compressing a plurality of files, or compressing a very large file, the compressed file will still be quite large. The compressed file needs to remain unchanged in the storage device (such as hard disk or tape) during the decompression process. Therefore, in order to decompress a compressed file with size S, double S size storage is required before finishing the decompression. This storage requirement is difficult to be satisfied for a large compressed file.
Several solutions for reducing the required storage for decompression have been proposed in existing technology. One solution is to delete the compressed data after the decompression is completed. A typical example of this solution is bunzip2 in Linux. By default, it will automatically delete the compressed file after its decompression is completed.
The second solution is to selectively decompress the compressed file. WINRAR® in Windows® operating system is an example of this solution. It allows the user to browse the files in a compressed data without decompressing it. Furthermore, if one or more files in the compressed file is selected and opened, the decompression will be started directly.
In the aforementioned two solutions, decompressed files are stored in the storage device. However, for the data in some special fields, such as banking transaction data, monitoring data of petroleum industry, meteorological data, scientific computing data, multimedia data, etc, the size of data are extremely large, as well as the compressed backup files. For example, a 100 G bytes database in the seismic field is very common. Such extremely large compressed backup files would need very large storage when decompressing, without considering the storage used by the decompressed file itself.